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(57) Abstract 

Disclosed is a low-cost fingerprint capture device 
that can be integrated in a computer mouse. Fingerprint 
information captured with such a device is transmitted to 
a computer via a serial port. As such ports are relatively 
slow, the capnire device ptovides a low resolution 
fingerprint image (having less than 512 x 512 pixels). 
This low resolution data is processed by an iterative 
method that contrasts sectors of the image in a direction 
specified by "flow vectors". These flow vectors specify 
an average direction of fingerprint ridgeiincs in the sector 
under consideration. These methodologies enable the 
transmission of fingerprint data over the relatively slow 
RS232 serial protocol, in a manner that allows virtually 
instantaneous fingerprint identification. 
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Embeddable Module for Fingerprint Capture and 

Matching. 

Cross Reference to Re lated Ap plirn^^^^., 

This application claims priority from US Provisional Patent 
Application No. 60/025,949, filed September 11, 1996 entitled 
EMBEDDABLE MODULE FOR FINGERPRINT CAPTURE AND 
MATCHING, and naming R. Rao. S. Subbiah. Y. Li, and D. Chu as 
inventors. That application is incorporated herein by reference in its 
entirety and for all purposes. 



Backyroun^ n f the ^nvy nfi^in 

This invention relates to fingerprint recognition methods and 
apparatus. More particularly, the invention relates to authentication 
systems that capture fingerprints, transmit fingerprint data through 
simple serial ports such as RS232 ports, and perform audientication in 
real rime based upon the fingerprint data transmitted through the serial 
ports. . 

Today, security is a prime issue of computer networking and 
computing in general. In the age of global connectivity and internet 
information exchange, authentication and or authorization of users is 
essential for the protection of both users and servers. The requirement 
for security includes the protection of sensitive personal files on a 
shared computer system, protection of personal information such as 
credit card numbers through the Internet for purchasing/protection, etc. 

Various encryption methods exist for dealing with such problems 
See for example. U.S. Pat. No. 5,373,559 (Kaufman et. al.). Most of 
tliese methods involve either passwords which are often difficult for 
users to remember or easy for computer hackers to crack, or some 
more complicated algorithms that in many cases require users to carry a 
"calculator" that would generate a new key at each time entry to a 
system is desired. 
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In many cases, it would be preferable to use biometric 
information, in particular fingerprints, for security purposes, since 
these are unique, do not require memorization on the part of the user, 
and are difficult to duplicate. Fingerprint technology including 

5 hardware image capture, software image processing, software/hardware 
for fingerprint data storage and software for fingerprint 
analysis/comparison is a relatively mature technology with over 20 
years of prior art methodology (See for example, U.S, Patents Nos. 
2,952, 181, 4, 15 1 ,5 1 2, 4,322, 163, 4,537,484, 4,747, 147, and 

10 5,467,403). It is well-known that no two individuals possess the identical 
fingerprint and that accurate matching techniques in conjunction with 
well-captured images can positively identify an individual. 

Hardware for fingerprint capture and software algorithms for 
subsequent fingerprint searching, matching, authentication and 

15 verification are available from many vendors for high-security purposes 
(e.g. from Identix of Sunnyvale, California; Digital Biometrics of 
Minnentonka, Minnesota; Printrack of Los Angeles, California; Morpho 
of France), and are routinely used by the police and more recently by 
other governmental entities for, e.g., verifying the identity of social 

20 welfare recipients and motorists registering with the Department of 
Motor Vehicles. The complete hardware/software systems require 
sizable investments, ranging from a few thousand dollars to many tens 
of thousands of dollars (larger more sophisticated systems can run into 
millions). The hardware capture devices are generally bulky — 

25 approximately the size of a standard brick ~ and need to be attached to 
a computer via an additional board that usually connects to the system 
bus. These boards are usually full-size or half-size boards of at least 
four inches by four inches and have to be specially installed within the 
computer housing and peripheral interface. 

At present, a low cost (consumer market pricing of significandy 
less than 100 dollars), miniaturized (about the size of the finger itselO> 
and easily usable fingerprint-based protection of user identity, privacy, 
and computer data - particularly for internet commercial applications - 
is not available for the general home/individual consumer/personal 
computer owner. The lack of such a low-cost, extremely small, and 
readily usable fingerprint-based security device is a significant 
constraint that prevents widespread everyday use of fingerprints. 

2 
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Summary n f thft Invonfinn 

The present invention provides a low-cost fingerpri nt capture 
devic e that can b e integrated in or tofailv emhpriHpfi >uithin i/r.rio„s 
exi sting computer nerinhcr ats. Fingerprint information captured with 
such a device may be used for authorization to use computer systems, 
for encryption of computer files and directories, for 
authorization/authentication when accessing remote computer 
stations/peripheral equipment including those connected by tlie 
internet/world wide web, and for general verification/authentication of 
the identity of the user. The hardware computer perip hp-rnjg suitable 
f or such integ ra tion include various types of existing mouse, plideparis, 
tr ackballs. icevboa rds.-and- orhp.r rr.\:i t^A mechanisms where a user 
controls access or inputs data tp a computer. 

Since many of these peripherals communicate with the computer 
through serial ports or relatively slow parallel ports, the transmission of 
the captured fingerprint information is a bottleneck to real-time (i.e. 
instantaneous, under a second) fingerprint verification. The present 
invention introduces advanced methodologies which enable the 
transmission of such information, in particular over the slow RS232 
serial protocol, in a manner that allows virtually instantaneous 
fingerprint identification. 

One aspect of the invention provides an apparatus for identifying 
a user's fingerprint. This system may be characterized as including the 
following features: (a) a fingerprint capture module which generates an 
image of the user's fingerprint in a low resolution format comprised of 
fewer than 512 x 512 pixels (thereby allowing rapid transmission over 
the RS232 format); (b) a connection line connected to the fingerprint 
capture module and capable of transmitting the low resolution 
fingerprint image; and (c) a processor connected to the connection line 
and configured to determine relative positions of minutiae in the low 
resolution fingerprint image. 

To generate the low resolution fingerprint image, the fingerprint 
capture module preferably includes a CCD array, a CMOS 
photodiode/photogate array, an imaging capacitor array formed on a 
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semiconductor substrate, or an imaging ultrasonic mechanism. The low 
resolution fingerprint image preferably comprises at most 240 x 240 
pixels, more preferably at most 160 x 240 pixels, and most preferably 
at most 160 x 120 pixels. To ensure real time fingerprint processing, 
5 the connection line preferably transmits such low resolution fingerprint 
image in not more than about two seconds. 

Another aspect of the invention provides a method of identifying 
a user's fingerprint. Tliis method may be characterized as including the 
following: (a) obtaining an image of a fingerprint; (b) generating a 

10 plurality of "flow vectors" from the fingerprint image; (c) contrasting 
the fingerprint image using the flow vectors to generate a flow 
contrasted fingerprint image; and (d) extracting fingerprint minutiae 
from the flow contrasted fingerprint image. Each of the flow vectors 
specifies an average direction of ridgeline flow at a sector in the image. 

15 Preferably, the method also "thins" the flow contrasted fingerprint 
image prior to extracting the fingerprint minutiae. 

Often steps (b) and (c) are repeated until the flow contrasted 
fingerprint image converges. In such an iterative procedure, a step of 
thinning the flow contrasted fingerprint image may be performed in 
20 each iteration. 

These and other features and advantages of the invention will be 
described in detail below with reference to the associated figures. 

Brief Description of the Drawings 

25 Figure lA is a table listing some of the signals transmitted on 

various pins of an RS232 connector 

Figure IB is an illustration depicting die connections for two 
serial ports communicating according to the RS232 format. 

Figure IC is a timing waveform illustrating transmission of 
30 signals according to the RS232 format. 

Figure ID is an illustration depicting the pin arrangement on an 
RS232 connector. 
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Figure 2A is a side sectional view of a fingerprint capture module 
m accordance with one embodiment of the present invention. 

Figure 2B is a top sectional view of the fingerprint capture 
module shown in Figure 2A. 

Figure 3 is a side sectional view of optics that may be employed 
in one embodiment of a fingerprint capture module of this invention. 

Figure 4A is a top view of a computer mouse outfitted with a side 
mount fingerprint capture module suitable for use with this invention. 

Figure 4B is a top view of a computer mouse outfitted with an 
internal mount fingerprint capture module suitable for use with this 
invention. 

Figure 5 is a flow-diagram representative of a conventional 
process employed in fingerprint processing software. 

Figure 6 is a thinned image of a low resolution (160 x 120 pixel) 
fingerprint image and processed by conventional fingerprint recognition 
software such that the thinned image has an unacceptably high number 
of false fusions. 

Figure 7 is a flow diagram depicting a fingerprint minutiae 
extraction technique that may be employed with low resolution 
fingerprint images of the present invention. 

Figure 8 is an illustration comparing thinned images derived 
from high and low resolution images of a fingerprint. 



Dgt^Ugd nftsrrintion of thP Prpf^^rf ^i Kmhndimap ^c 

For purposes of diis discussion, the term "computer" will 
generically refer to all computing devices, including personal 
computers, network computers, bridges, routers, work stations, 
supercomputers, embedded computer processor chips, etc. 
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The term "fingerprint" as used herein refers to handprints, 
palmprints, and other unique skin patterns in addition to traditional 
fingerprints. 

For convenience, most of the peripherals discussed herein will be 
5 referred to generically as "generic computer mouse" or simply 
"mouse." These include, for example, various types of existing mouse, 
glidepads, trackballs, keyboards, and other related mechanisms where a 
user controls access or inputs data to a computer. 

The discussion below will present certain preferred apparatuses 
10 and methodologies for implementing the present invention. First, 
certain information setting forth preferred environments of the 
invention will be described. These include serial ports (particularly 
those adhering to the RS232 format) and computer processing of 
conventional fingerprint data. It should be understood that many details 
15 discussed herein are provided simply to help illustrate the invention and 
are not necessarily intended to limit the invention. Thus, the invention 
may be practiced without limitation to some or all of the details set 
forth to illustrate preferred embodiments. 

It should be understood that the present invention also relates to 
20 machine readable media on which are stored software produced in 
accordance with the requirements of this invention, or program 
instructions for performing methods of this invention. Such media 
include, by way of example, magnetic disks, magnetic tape, optically 
readable media such as CD ROMs, semiconductor memory such as 
25 PCMCIA cards, etc. In each case, the medium may take the form of a 
portable item such as a small disk, diskette, cassette, etc, or it may take 
the form of a relatively larger or immobile item such as a hard disk 
drive or RAM provided in a computer. Furthermore, program 
instructions implementing this invention may sometimes be transported 
30 over a communications network. 



A. Serial Ports 

As known in the state of the art, there are two types of standard 
ports on computers, including personal computers, that are used to 
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communicate data with peripheral components: one is the parallel type 
of port which uses eight separate lines to transmit data in parallel, and 
the other is the serial type of port/interface which uses a single line. As 
they are physically larger, and occupy more external surface area, there 
is typically only one parallel port available on a computer. Serial ports, 
however, are smaller and often more than one is present on a computer. 
Serial ports come in a variety of formats of which tfie most common is 
the standard RS232 type that was developed in the early days of 
telephone communication. 

Another type of serial port that is common on a computer is the 
PS/2 serial port which is originally developed by IBM Corporation of 
Armonk, NY. Today, some computers, particularly personal computers 
that are IBM-compatible, have them. 

The higher-capacity parallel port is typically used by printers, 
some scanners, digital cameras, and other peripheral components that 
deliver large quantities of information to the computer. The lower 
capacity serial ports arc typically used by less data-intensive computer 
control/data input peripherals of the "generic mouse" and telephone 
modems. In particular, on IBM-compatible personal computers, the 
PS/2 type of serial port is solely dedicated to service the computer 
mouse and keyboard. On many IBM-compatible personal computers a 
computer mouse can be attached through either the PS/2 serial port or 
alternatively through additionally available RS232 serial ports. On non- 
IB M-compatible personal computers, any of the available RS232 ports 
can be used by the computer mouse. 

With the serial port interface, where the RS232 data transmission 
format is virtually a de-facto standard, the constraints of efficiently 
transmitting fingerprint information requires special consideration that 
the present invention overcomes. Even when a faster transmission 
format is employed, the present invention allows fingerprint recognition 
to be performed in a manner that consumes less computer resources. 

Figures la to Id illustrate the RS232 serial communication 
protocol established by the Institute of Electronic and Electrical 
Engineers (IEEE) as an international standard for telephone 
communication. As known in the state of the art, the standard allows 
for specific pin numbers 101 (Figure la) on an RS232 connector 106 
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(Figure Id) to correspond to electrical wires attached to it that 
specifically carry out the many different functions 112 (Figure la) 
required for efficient communication between two devices 111 using the 
RS232 serial protocol. The wires are of two broad types — control 
5 lines and those that are specifically used to transmit data, like 
fingerprint image data. For example pin number 2 (102) carries 
information transmitted in one direction 108 while pin number 4 (104) 
requests the data to be sent, and pin number 7 (107) is the signal 
ground. 

10 Figure lb illustrates how some of these electrical wires are 

physically attached to either end of the serial communication link III. 
For instance, the signal ground 110 of the connectors at either end are 
wired together while the electrical line for transmitted data 105 from 
the transmitting connector is wired to the electrical line for received 

15 data on the receiving connector 109 and vice versa. 

Figure Ic illustrates the actual scheme for packaging the bits of 
information to be sent/received using the transmit 102 or receive 103 
electrical lines. As is explained in detail on pg. 477 of Horowitz and Hill 
(The Art of Electronics. 1980, Cambridge University Press), hereby 

20 incorporated by reference for all purposes, anywhere between 4 to 8 
data bits can be sent in a single packet which can be additionally framed 
by one start bit and one stop bit. The packets can be transmitted at 
various allowed baud rates (bits per second) that ascend as follows — 
110, 150, 300, 600, 1200, 2400, 4800, 9600, 19,200, 38,400, 57,600, 

25 1 15,200. Typically, even the latest personal computers do not allow for 
sustained RS232 serial transmission rates higher than 1 15,200 baud. 



B. Reducing the Quantity of Transmitted Fingerprint Image 

Data 

30 When higher rates of data transmission are required by a 

peripheral, the data has to be transmitted directly to the bus through 
either a special bus port or more commonly through a dedicated board 
that has to be inserted into a computer. Fingerprint image data which is 
highly voluminous, is currently sent through a dedicated board that 

35 connects dii"ectly to the bus. 
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The present invention alleviates the inconvenience that users of 
captured fingeiprint information face when forced to install a dedicated 
computer board/card onto the mother board bus itself. One 
embodiment of the present invention is directed to the use of the 
external parallel port (without a special card) for obtaining captured 
fingerprint information from a computer peripheral. 

Conventionally, the RS232 serial ports and even slower PS/2 
ports are not used for fingerprint recognition because the maximum 
rate of serial data that can be handled by present-day computers 
(typically using the 16550 UART chip which is well known is the state 
of the art) is only 115,200 bits per second (baud rate). The time 
required to send a fingerprint image that is not specially processed for 
the RS232 standard (i.e., not employing the techniques of the present 
invention) is in the few tens of seconds at the maximum baud rate 
presently available. While the parallel port is typically several time 
faster than the serial port, the required time is still more than a few 
seconds - at least without the special processing that the present 
invention provides. 

Even if the faster parallel port is used, fingerprint capture 
peripherals provided through such parallel port would not be the ideal 
solution. Note that a typical personal computer has only one parallel 
port and many competing peripherals like printers, scanner etc. to 
occupy it. Often no such competition exists for serial ports; frequently 
there is at least one spare RS232 serial port available. The present 
invention has the advantage of being suitable for use with an external 
serial port, in particular those which use the slow but common RS232 
serial format, to transmit captured fingerprint information. 

Still further, the present invention provides the use of the low- 
capacity RS232 format itself to deliver the voluminous data normally 
associated with the fingerprint images. Because the physical data 
transfer-rate is slow, the present invention preferably employs special 
handling (described below) of the fingerprint data to accommodate the 
RS232 format and sends the information very rapidly (typically on the 
order of a second or two). 

Such efficient use of the slow RS232 format is facilitated by a 
biometric processing routine that we have developed. For real-time 
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instantaneous fingerprint-matching, the RS232 protocol is too slow to 
transmit fingerprint images at a resolution sufficiently high for image 
processing by traditional fingerprint software. In order to achieve 
rapid transmission over RS232 serial lines the current invention allows 

5 capture of a relatively coarse and significantly low-resolution 
fingerprint image. This limits the amount of data that must be 
transmitted over the serial port. It also limits the quantity of processing 
resources that must be deployed to process the fingerprint data. Thus, 
fingerprint recognition can be performed in real time even when the 

10 fingerprint is captured on a device connected by a relatively slow serial 
port. 

In prior recognition systems such advantages were not attainable 
because far higher resolution data was required to perform accurate 
fingerprint matching. In the present invention, the transmitted 

15 fingerprint image may be rather crude in comparison to that typically 
processed by prior art algorithms (see for U.S. Pat. No. 2,952,181, 
4,151,512, 4,322,163. 4,537,484, 4,747,147, 5,467,403 and NIST 
manuals). Thus, an important aspect of tlic current invention is a 
methodology that we have developed to overcome the crudeness of the 

20 transmitted fingerprint data and restore higher resolution features of the 
original fingerprint. In effect the lower resolution of the transmitted 
print, while allowing rapid transmission, introduces ambiguities in the 
details of the fingerprint image, diat prior art methods cannot restore. 
Since these details are crucial to subsequent accurate fingerprint 

25 comparison, the current invention, unlike the prior art, provides a 
special methodology (described below) for correctly resolving these 
ambiguities prior to traditional fingerprint comparison. Therefore, the 
methods of this invention allow for the rapid and efficient use of the 
RS232 serial protocol, whose speed was heretofore inadequate, to 

30 transmit captured fingerprint information and allow real-time 
fingerprint comparison. 

At the currently available maximum baud rates (which represents 
the bulk of the installed base of computers well into the next century) of 
115,200 bits per second, the transmission of the standard resolution 
35 fingerprint data is much too slow. The 'Federal Bureau of Investigation 
(FBI) and National Institute of Standards (NIST) approved* fingerprint 
images require 512 x 512 = 262,144 individual pixels each with 256 
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levels of gray shades (represented by 8 bits), would take some ( 262 144 
X 8 )/ 115,200 = 18.2 seconds. This is haidly real-time and is 
unacceptable for the envisaged swift and interactive consumer uses that 
low-cost fingerprint technology can be widely put to. 

For fingerprint technology to take-off with average consumers, 
fingej print image transmission should occur on the order of one second. 
The present invention achieves such a real-time transmission on 
standard RS232 lines by two means, one hardware and the other 
software. Further, it is the particular marriage of these two means that 
allows this rapid transmission possible. First, lower resolution and 
therefore coarser fingerprint images - in one embodiment 240 x 160 = 
38 400 pixels and in a further preferred embodiment only 160 x 120 = 
19 200 pixels - are transmitted over the RS232 serial line. This can be 
done by using CCD or CMOS photodiode/photogate cameras with 
suitably lower number of pixels. However, cameras with such non- 
standard pixel numbers are relatively expensive compared widi the 
standard ones mass produced for the multimedia/video market like 512 
X 512 and 320 x 240 pixels. Therefore one aspect of our invention 
requires the adaptation of these higher-resolution cameras to only send 
alternate lines of pixels and within these lines only every other pixel. In 
general, die invention applies to any technology in which an image of 
lower resolution tlian the standard 512 x 512 pixel image (i.e., an image 
containing less than 262,144 pixels) is transmitted from a computer 
peripheral for further processing. 

Suitable haidware logic circuitry allows the 512 x 512 camera to 
send only 240 x 240 pixels and the 320 x 240 camera to send only 160 x 
120 pixels. In another embodiment, the 320 x 240 camera can be 
arranged to only send out one contiguous half of the pixels, so that the 
camera window is now half at 160 x 240 pixels. In a further 
embodiment only every other row of the 320 rows is dropped to give a 
different type of camera window that is also 160 x 240. However, a 
preferred embodiment adapts the 320 x 240 camera for use as the really 
coarse 160 x 120 camera mentioned above. 

Another aspect of tliis data reduction whereby higher resolution 
fingerprint data is dropped in favor of quicker transmission of 
fingerprint images is the adoption of gray levels that are coarser than 
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the standard 256 shades. In one preferred embodiment, the system 
employs a coarser 64 gray scale that can be encoded in 6 bits, in 
combination with the lower number of pixel results in complete but less 
detailed fingerprint images that can be contained in as little as 160 x 120 
5 X 6 = 115,200 bits of data. Such relatively crude images can be 
transmitted on a RS232 line operating at 1 15,200 baud in about 1 second 
(see Figures I a- Id for further details; see also pg. 477-78 of Horowitz 
and Hill, The Art of Electronics. 1980, Cambridge University Press) 

While tlie real-time (about I second) constraint for fingerprint 
10 data transmission can now be met, the crudeness of the images causes 
problems in interpreting and analyzing these fingerprint images 
received by the computer. In particular, a 160 x 120 image is at or even 
beyond the theoretical limit required for preserving the unique patterns 
of fingerprint ridgelines that define and so differentiate one human 
15 being from another. 

A typical human fingerprint has a aspect ratio of 3 to 2 (i.e., it is 
1/2 time as long as it is wide). The average fingerprint has about 50 to 
60 ridgelines separated by intervening valleylines that are about equally 
as thick. Generally the lines run from left to right and as they do they 

20 Fust traverse upwards and later downwards. Even if these lines were 
horizontal and did not traverse longitudinally, and each ridgeline was 
one pixel thick and each valleyline was also one pixel thick, one would 
need at least 60 x 2 = 120 pixels in the longitudinal direction to clearly 
resolve all the lines. This is the reason for the FBI-approved choice of 

25 512 x 512 pixels since it allows for at least four pixels per ridgeline and 
four per valleyline. With the stringent choice of 160 pixels, as in a 
preferred embodiment, allowing for tlie generally curved and non- 
horizontal ridgelines would still be pushing the limit for resolving truly 
independent ridgelines as being separate. That is to say one will 

30 inadvertently generate many fusions between ridgelines that are 
spurious. 



C. Structure of a Fingerprint Capture Module 

Figures 2A and 2B show, respectively, side and top views of an 
35 exemplary arrangement of components on a fingerprint image capturing 
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module 200. Finger 201 is pressed on the surface 205 of the optical 
mount 203 which sits on top of an image capturing device such as a 
CCD camera 204 (or a CMOS photodiode/photogate camera). There is 
a light source 202 that uniformly illuminates the finger 201. The light 
source 202 could be a single LED which is part of a larger diffuser that 
would provides uniform lighting over the entire finger 201 at the 
surface 205. The circuit board 206 is preferably smaller than a one or 
two square inches in area. The cable 207 extends from the circuit board 
206 to the RS232 serial communication port. For clarity, electronic 
components, other than CCD 204, on the circuit board 206 are not 
shown. Such other components include, for example, RS232 drivers, 
analog-to-digital converters, power supplies, clocks, FPGA controllers, 
and timers. 

Figure 3, illustrates the details of an exemplary optical and 
lighting apparatus. Lighting source 202 could be a single LED which 
can be powered from the circuit board 206. The optical mount 203, 
which securely fits over the CCD 204, is enlarged and shown in detail' 
It can be made from a single molded piece of acrylic material. The lens 
301 is shown to be part of the optical mount 203. The surface 205 is 
where the user presses his/her finger 201. The optical mount 203 
allows light to travel from the illuminated finger 201 to die CCD 
camera 204. 

In an alternative embodiment, the fingerprint capture module 
may be an imaging capacitor array formed on a semiconductor substrate 
such as diat described in the May 22, 1997 edition of the San Francisco 
Chronicle, "New Chip Verifies Fingerprints" which pertains to a 
product of Veridicom Corporation. An advantage of this embodiment 
is that it does not require the optics for focusing an image of the 
fingerprint onto the CCD or CMOS array. In another alternative 
embodiment, imager 417 may be an ultrasonic mechanism formed on 
semiconductor substrates. 

Figures 4A and 4B shows two exemplary embodiments of a 
fingerprint capturing device that can be integrated/embedded into a 
generic o r^common computer mouse 401^ The generic mouse 401 has 
the typical comp onents of a mouse such as nip use"button(s) 4027 The 
cable 207 to the^szsJ. serial port can transmit the data from both the 
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mouse 401 and the circuit board 206 of the fingerprint capturing device 
that is enibeddedjnjh^^ l . l he particular embodiment shown 

in Fi^re 4B only requires a small opening to be carved from the side 
of the mouse where finger 201 could be pressed onto the surface 205 of 
the optical mount 203. The embodiment shown in Figure 4A requires 
the molding of the common mouse 401 to be slightly enlarged on one 
side. This embodiment is intended to fit the shape of a hand 
comfortably. Although not shown, this fingerprint capturing 
device/module can be integrated easily into other serial peripheral 
components such as glidepads, trackballs^ and keyboards. 

In a further refinement of the present invention, the fingerprint 
module itself can include an on-board embedded computer processing 
chip that can at least partly process the captured fingerprint image 
locally. Such partly processed fingerprint information from a 
fingerprint capture peripheral can be sent for further 
processing/interpretation to either another computer typically larger 
connected to it or other electronic peripherals using the same RS232 
serial telephony format itself. Therefore, the present invention also 
provides the use of the RS232 format for transmitting partly processed 
fingerprint information captured by the fingerprint capture device 
between computers (including embedded computers on-board the 
fingerprint capture device itselO and all peripheral equipment connected 
to them by such a RS232 serial format. In some embodiments, the 
capture module uniquely identifies the fingerprint by extracting 
minutiae (described below), while the processing module matches the 
newly captured fingerprint against one or more stored fingerprints. 

It should be understood that while many benefits of the invention 
can be realized when implemented on a peripheral connected over a 
serial port, the invention can also be deployed in devices connected to 
parallel ports. 



D. Minutiae Extraction from High Resolution Fingerprints 

Accurate fingerprint matching technology, which is well-known 
in the art (see, for example, U.S. Patent Nos. 2,952,181, 4,151,512, 
4,322,163, 4,537,484, 4,747,147, and 5,467,403 which were previously 
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incorporated by reference), has for over a hundred years relied on the 
extraction and subsequent comparison of specialized features called 
minutiae. Minutiae are essentially of two equally frequent types - either 
the abrupt ending of a line in the middle of the fingerprint or the fusion 
of two lines to create a Y-shaped junction. Typically there are about 60 
or 70 such features in a fingerprint and it is the relative location of 
these from each other that creates a unique spatial pattern that 
statistically no other human can possess. 

Traditional methods of fingerprint matching may involve 
software processing steps as illustrated in Figure 5. After capturing the 
fingerprint image (step 501) with an FBI-standard 512 x 512 pixel 
array, a contrasting step (step 503) reduces all die gray shades of a 
captured image 502 to either black (for ridgelines) or white (for valley 
lines) as shown in image 504. Traditionally these methods are omni- 
directional. That is, a priori, they have no notion of what direction the 
ridge lines are generally flowing at any given pixel. Basically, the 
particular shade of gray at each pixel is compared with those of the 
neighboring pixels in all directions and if judged to be relatively darker 
than most of its neighbors it is deemed to be black, otherwise white. 
For the purpose of contrasting the present invention from traditional 
methodology, a key point is that the judgment in traditional methods is 
made without any knowledge of the general direction of the overall 
ridge-flow in the fingerprint. 

After diis contrasting step, the contrasted image 504 is further 
processed by a thinning method (step 505). The object here is to reduce 
the black lines from being on average four pixels thick to only one pixel 
thick, thereby increasing the number of white pixels substantially. A 
thinned image 506 is then examined by further mediods (step 507) that 
attempt to deduce and accurately extract the minutiae and their locations 
as shown in a map 508. The process is then completed at 509. All 
further fingerprint matching/comparison relies only on these 60 or 70 
extracted pieces of information. Therefore the generation of false 
minutiae is particulariy damaging to their use as a means to uniquely 
identify individuals. 

When the high-resolution 512x512 captured image 502 as shown 
in Figure 5 is processed by traditional software as described, a nice 
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separation between ridgelines 803 (Figure 8) results in extraction of 
only true minutiae/features 508 and 804. 

Figure 6 provides an example of a low resolution 160 x 120 pixel 
fingerprint image that was image processed using the procedure of 
Figure 5. Remember that the procedure of Figure 5 is meant to apply 
to higher-resolution 512 x 512 images without benefit of the methods of 
this invention. Referring direcdy to Figure 6, note the extremely large 
number of false contacts/fusions between independent ridgelines, that 
thereby swamping the smaller number of real minutiae. 

Owing to the lack of resolution, when the traditional method 
(Figure 5) is routinely applied to a coarse 160 x 120 pixel image, the 
thinned image can be typically expected to look like a tangled mass with 
numerous false fusions overwhelming the true ones. This is to be 
compared widi the nicely thinned image 506 in Figure 5. Thus, 
reducing the resolution in order to send less pixel in the 1 -second rather 
than the I8-sccond time-frame down a RS232 serial line will 
traditionally result in a set of fingerprint minutiae that is so corrupted 
that it is worthless for the purpose of fingerprint-based identification of 
individuals. Thus, a mass market for fingerprint technology based on 
exploiting the widely installed RS232 serial line route (e.g. mouse or 
keyboard) appears untenable. 



E. Extracting Minutiae from Low Resolution Fingerprints 

Figure 7 is a modification of the traditional fingerprint image 
processing schemes illustrated in Figure 5 that includes two 
methodologies to allow coarser 160 x 120 images to be used to extract 
accurate minutiae without introducing false minutiae that may arise due 
to the low-resolution. In particular, a directional contrasting method 
703 and a flow-vector generating method 705 are used in a cyclical 
manner until nicely resolved thinned images (like those from 512x512 
images) are obtained even from 160 x 120 images. 

Together these methods allow the artificially generated false 
minutiae stemming from the coarse 160 x 120 images to be detected and 
removed, leaving only the true minutiae (i.e. ridgeline fusions). In an 
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initial cycle, ilie modified method proceeds as usual from the captured 
image 701, to an omnidirectional contrasting step 702, and to a thinning 
step 704. Then a new method step 705 calculates general How vectors 
over an entire thinned image 710. In a preferred embodiment, step 706 
is performed as follows. The thinned image is divided into relatively 
course square .sectors and a general preferred average direction of the 
ridgeline flow is computed for each sector. These tlow vectors are then 
temporarily stored. 

It has been found that this method works well if the fingerprint 
image is divided into at least about 100 sectors. Even better results can 
sometimes be obtained if the image is divided into between about 300 
and 500 sectors (400 sectors in a particularly preferred embodiment). 
Depending upon the number of pixels in the image array, each sector 
may include between about fifteen by fifteen pixels and thirty by thirty 
pixels, for example. 

In one embodiment, the flow vectors in each sector arc generated 
as follows. Each pixel in the sector is analyzed to generate its own flow 
vector. Then, the flow vectors for each pixel in the sector are averaged 
to obtain a sector flow vector. Note that each pixel's flow vector simply 
comprises a direction of arbitrary but uniform magnitude. Therefore, 
to obtain a sector's flow vector, the directions of all pixel flow vectors 
within the sector are linearlly averaged. The flow vector for a given 
pixel may be obtained as follows. The pixels surrounding the pixel 
under consideration are each considered to determine which of them 
contain "ink" and which do not. From this, a direction of the ink can be 
obtained, and that direction defines the flow vector for the pixel under 
consideration. In one approach, only those pixels in the two nearest 
rings surrounding the pixel under consideration arc considered in 
generating the local flow vector. 

Unlike traditional methods, this method does not immediately 
proceed lo minutiae/feature extraction (step 708). Rather, it cycles 
through directional contrasting (step 703), thinning (step 704). and flow 
vector generation (step 705) until the flow vectors converge. Each 
time, die original captured image (provided in step 701) is contrasted 
with the newly generated flow vectors. Only on the initial cycle is 
omnidirectional contrasting (step 702) performed. 
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Each time the new flow vectors are generated, the system 
compares the newly generated set of flow vectors against the set of flow 
vectors from the previous cycle at a step 706. Alternatively, step 706 
could be performed by comparing the newly thinned image against the 
thinned image from the previous cycle. If the thinned image is 
improving (as determined at decision step 706), process control is 
directed to a step 703 where a novel contrasting method turns the gray 
shaded image into a black/white one in a directional manner. At each 
pixel, the local general ridgeline flow vector that was previously 
temporarily stored for the vicinity of a sector under consideration is 
used to selectively darken or lighten pixels in that direction. Instead of 
using neighboring pixels in all directions as a baseline for assessing the 
darkness or lightness of the pixel, pixels along the ridge flow vector 
direction as well as pixels along a direction completely orthogonal to it 
are used to make the judgment. This directionally contrasted image Is 
then thinned as normal at step 704. Again the new ridge flow vectors 
are calculated over all sectors at step 705. If the thinned image appears 
to be improving 706, the entire cycle of directional contrasting 703, 
thinning 704 and flow vector generation 705 is repeated until 
convergence as determined at step 706. 

One approach to specifying convergence is as follows. All flow 
vectors from the current iteration are compared against those from the 
immediately previous iteration. If the average change in flow vector 
direction (averaged over all sectors of the image) is less than a defined 
amount (5' for example), then the procedure is deemed to have 
converged. In an alternative embodiment, if the average change in flow 
vector direction is less than the average standard deviation (average 
over the standard deviations for each sectors), the image is deemed to 
have converged. Note that the standard deviation for a given sector is 
obtained by considering the local flow vectors for all pixels making up 
that sector. 

When the method finally determines at step 706 that the image has 
converged, process control is directed to a step 708 where the minutiae 
are extracted. The procedure is then completed at a step 709. The net 
result of this procedure is a thinned image of even 160 x 120 pixel 
coarse images that are substantially free of artefactual false fusions 
stemming from the lack of sufficient resolution in the captured image. 
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This process of removal of false fusions is illustrated in more 
detail in Figure 8. In a portion of a 512 x 512 fingerprint image 802, 
well separated ridges are obtained. The one true Y-shaped fusion 804 is 
clearly found correctly in the thinned image where the thinned lines 803 
are nicely resolved. When the same portion of the same fingerprint is 
captured at the lower 160 x 120 image resolution 806. one can clearly 
see the ill-effects of the use of coarser pixels - ridgelines that are so 
close to each other that any two parallel lines inadvertendy touch each 
other at many points along their lengths. When this coarser image is 
thinned by traditional methods, these spurious points of contact lead to 
multiple false fusions that swamp the one real fusion in a relatively 
useless thinned image 807. Nevertheless, when instead die above 
method is applied (indicated by arrow 808) to the coarser 160 x 120 
image 803, it is possible to obtain the well-resolved thinned images, 
showing the only one true fusion 804, characteristic of the higher- 
resolution 512 X 512 image. 

After the minutiae have been extracted, they are typically 
matched against the stored fingerprints. This requires matching the 
two-dimensional coordinates of the stored and recently captured 
fingerprints. If the coordinates match to within a defined tolerance, the 
tokens are deemed a match. 

As known in the state of the art, many fingerprint matching 
schemes involve die generation of inter-minutiae-based keys (i.e., 
distance vectors, etc.) that while being generally similar, will vary 
between multiple impressions of the same finger. Various inter- 
minutiae distance-vector-derived formats are known in die art. Many 
of these (as well as variations on them) may be suitable for use with this 
invention. 

Suitable matching schemes arc described in, for example. US 
Patent No. 4,747,147 issued to Sparrow on May 24, 1988, US Patent 
No. 5,493,621 issued to Matsumura on February 20, 1996, and 
information provided at the World Wide Web site 
www.Lucent.Com/Press/0597/minul.GAF. Each of these documents is 
incorporated herein by reference for all purposes. A typical description 
of a processed fingerprint is a list of x, y and angle tabulation of each 
minutia. Minor modification to these linear values (e.g., adding slight 
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random displacements) will still reflect the same underlying fingerprint, 
allowing for variation during multiple impressions (e.g., slight 
distortions and rolling during the pressing of the finger). 

In conclusion the introduction of the above-described method 
5 allows coarse fingerprint images, that would be normally useless in the 
context of prior art minutiae extraction methods, to be nevertheless of 
use for fingerprint recognition. This ability to use such low-resolution 
data enables exploitation of the slow but overwhelmingly common 
RS232 serial line to deliver real-time fingerprint comparison to the 
10 mass consumer market. 



F. Other Embodiments 

In general, the present invention provides techniques which allow 
rapid processing of fingerprint images to extract minutiae with 

IS relatively little expenditure of computational resources. Such 
techniques may be deployed in numerous applications. For example, 
they may allow comparison of stored fingerprint data with a user's 
fingerprint data taken at the time of installation or operation of a 
software program. Such methods may be employed for the purpose of 

20 controlling software distribution. Various methodologies employing 
such comparison may be used in this invention. Some of these 
methodologies are described in US Patent Application Serial No. 

08/ , (Attorney Docket No. HUSHP002). entitled A BIOMETRIC 

BASED METHOD FOR SOFTWARE DISTRIBUTION, naming S. 

25 Subbiah, D. R. K. Rao, and Y. Li as inventors, and filed on the same 
day as the instant patent application. That application is incorporated by 
reference for all purposes. 

Another potential use of such a system involves giving the 
software manufacturer the option of charging its clients by usage — 
30 either by usage time or number of users. That is, the concept of pay- 
per-use. If the original user or buyer is given the option of adding 
additional users by the program, then the program can keep track of the 
number of users and the amount of elapsed time at each use. Such 
information can be sent back to the manufacturer periodically, cither by 
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postal mail for the first method or electronic mail for the second 
method, for billing purposes. 

In another embodiment, this invention is employed in a system 
and method employing a user's fingerprint to authenticate a wireless 
communication. The user's personal fingerprint is employed as the 
secret key in the context of a modified "challenge-response" scenario. 
The system includes a fingerprint capture module on a mobile personal 
wireless communicauon device (e.g., a wireless telephone) and a central 
authentication system coupled to a conventional mobile switching center. 
The central authentication system contains information that associates 
each mobile identification number ("MIN") with a particular user's 
fingerprint. When a wireless communication is to be initiated, the 
central authentication system engages in a challenge-response 
authentication with the mobile switching station or the wireless phone 
using the stored fingerprint associated with the MIN through the 
common air interface. The correct response from the mobile station 
will only be generated when the user's fingerprint entered through the 
fingerprint capture module attached to the mobile station matches the 
information sent from the central authentication system, and only calls 
placed from authorized users are connected. This embodiment is 

described in US Patent Application Serial No. 08/ (Attorney 

Docket No. HUSHP003), entitled METHOD OF USING FINGERPRINTS TO 
AUTHENTICATE WIRELESS COMMUNICATIONS, naming Y. Li. D. R. K. 
Rao, and S. Subbiah as inventors, and filed on the same day as the 
instant patent application. That application is incorporated by reference 
for all purposes. 

While this invention has been described in terms of a few 
preferred embodiments, it should not be limited to the specifics 
presented above. Therefore, the invention should be broadly 
interpreted with reference to the following claims. 
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CLAIMS 

What is claimed is: 

1. An apparatus for identifying a user's fingerprint, the 
system comprising: 

a fingerprint capture module which generates an image of 
the user's fingeiprint in a format comprised of fewer than 512 x 512 
pixels; 

a connection line connected to the fingerprint capture 
module and capable of transmitting the fingerprint image in the format 
comprised of fewer than 512 x 512 pixels; and 

a processor connected to the connection line and configured 
to determine relative positions of minutiae in the fingerprint image. 

2. The apparatus of claim I, wherein the fingerprint capture 
module includes a CCD array or a CMOS photodiode/photogate array. 

3. The apparatus of claim 1, wherein the fingerprint capture 
module includes an imaging capacitor array formed on a semiconductor 
substrate or an ultrasonic mechanism formed on a semiconductor 
substrate, 

4. The apparatus of claim 1, wherein the fingerprint capture 
module provides the fingerprint image in a format comprising 240 x 
240 pixels. 

5. The apparatus of claim 1, wherein the fingerprint capture 
module provides the fingerprint image in a format comprising 160 x 
240 pixels. 

6. The apparatus of claim 1, wherein the fingerprint capture 
module provides the fingerprint image in a format comprising 160 x 
120 pixels. 

7. The apparatus of claim 1, wherein the connection line is 
capable of transmitting the fingerprint image in not more than about 
two seconds. 

8. The apparatus of claim 1, wherein the connection line 
transmits the fingerprint image in the RS232 format. 

9. The apparatus of claim 1, wherein the processor is 
provided in a computer and the fingerprint capture module is provided 
in a computer peripheral. 
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10. The apparatus of claim 9, wherein the computer peripheral 
IS a mouse, a trackball, a keyboard, or a glidepad. 

11. Tlic apparatus of claim I, wherein Uic processor is 
provided in a cellular telephone. 

12. The apparatus of claim I, wherein the processor contrasts 
and Ihins the fingerprint image prior to determining tlic relative 
positions of the minutiae in the user's fingerprint. 

13. The apparatus of claim 1, wherein the processor contrasts 
the fmgcrprint image with a plurality of flow vectors from the 
fingerprint image, each of the flow vectors specifying an average 
direction of ridgeline flow at a sector in the image. 

14. A metliod of identifying a user's fingerprint, the method 
comprising: 

(a) obtaining an image of a fingerprint; 

(b) generating a plurality of flow vector^i from the 
fingerprint image, each of the flow vectors specifying an average 
direction of ridgeline flow at a sector in the image; 

(c) contrasting the fingerprint image using the flow vectors 
to generate a flow contrasted fingerprint image; and 

(d) extracting fingerprint minutiae from the flow 
contrasted fingerprint image. 

15. The mediod of claim 14, further comprising: 

thinning the flow contrasted fingerprint image prior to 
extracting the fingerprint minutiae. 

16. The method of claim 14, wherein (b) and (c) are repeated 
until the flow contrasted fingerprint image converges. 

17. The method of claim 16, furtiier comprising a step of 
thinning the flow contrasted fingerprint image generated in each 
iteration. 

18. The method of claim 16, wherein the fingerprint image is 
provided in a format comprising 240 x 240 pixels. 

19. The method of claim 16, wherein the fingerprint image is 
provided in a format comprising 160 x 240 pixels. 

20. The method of claim 16. wherein the fingerprint image is 
provided in a format comprising 160 x 120 pixels. 
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21. The method of claim 16, wherein the fingerprint image is 
provided from a computer peripheral over a connection line that 
transmits the fingerprint image in an RS232 format. 

22. The method of claim 21, wherein the computer peripheral 
is a mouse, a trackball, a keyboard, or a glidepad. 



24 



wo 98/11501 PCTAJS97/I6092 



//6 




cc 

"<: 



CO 



<: 



O 
CO 



Q o 



CO 



o 
c 

-5* 

in 



c: 
o 

CD 



O 
Ol 



0> 

JQ 



CM 



CD 



00 



o 

UJ 

rsj 

I— I 
o 
o 



in 



CM 



Z CD 

o 
o 



r-i CO 



\ 



to to 



o 



I 



SUBSTITUTE SHEET (RULE 26) 



wo 98/11501 



PCT/US97/16092 





"^204 

SUBSTITUTE SHEET (RULE 26) 



wo 98/11501 



PCTAJS97/16092 




wo 98/11501 



PCT/US97/16092 




505 



507- 



509- 





^ Thinning ^ 












Minutiae/Feature 
Extraction 








End Program 





FIG. 5 

(PRIQRART) 




SUBSTmiTE SHEET (RULiE26ir 



wo 98/11501 



PCT/US97/16092 



5/6 



FIG. 6 




703 



To resolve ridge lines 
that touch eoch other 
(Ridge flow direction enhanced) 



704 



Thinning 



L 



705 



Generate flow vectors 
over iraoge 



NO 



L 



70S 



Convergence 
(Is the thinned luge iaprovin^) 



YES 



L 



708 



Minutiae/Feature 
Extraction 



L 



709 



End Program 



701 



Provide Captured Image 



Contrasting 
(omnidirectional) 



702 



f 



710 



^ — - - 



^ ^ ^ - * 
✓ / ^ • 

/ / ^ - 



FIG, 7 



SUBSTITUTE SHEET (RULE 28) 



wo 98/11501 



6/6 



PCT/US97/16092 




SUBSTITUTE SHEET (RUL£ 26) 



